PHILOSOPHY OF SCIENCE AND THE QUANTITATIVE-QUALITATIVE DEBATES: Thirteen Common Errors

One of the most important benefits of the qualitative-quantitative &bate in evaluation has been the increased awareness it has brought evaluators about philosophy of science. But evaluators are rarely philosophers, and consequently their presentations of philosophical material may contain errors. This article highlights thirteen common errors of this kind, and discusses some implications of these errors for the quantitative-qualitative debate. What are the most important accomplishments in the first 30 years of program evaluation? To me, the introduction of qualitative methods into evaluation should certainly be near the top of the list (Shadish, Cook, & Leviton, 1991). Without doubt, these methods have proven their utility to practicing evaluators, their distinctiveness to theorists, and their attractiveness to readers of evaluation results. Qualitative methods are here to stay, and evaluation is much better for it. In the process of introducing these methods, many theorists have also introduced arguments from philosophy of science that explain why these methods ought to receive more attention. This philosophical material is as welcome as the methods themselves, helping us to think more clearly and completely about why we do what we do. For example, as a result of these arguments, most evaluators are more aware of the socially constructed nature of evaluative knowledge, of the philosophical ambiguities that necessarily surround any scientific methodology, and of the near impossibility of justifying any particular method as being always and everywhere best. Unfortunately, very few evaluators are philosophers by training. Most evaluators learned about these matters either by reading what other evaluators say about philosophy of science, or more rarely by reading original philosophical works themselves. In my own work, I have learned from both sources. In the process, however, I have also learned that reading primary sources in philosophy is a far more reliable source of what philosophers say than is reading what evaluation theorists say philosophers say. The discrepancies are often substantial. Hence the goal of the present article is to correct 13 common errors about philosophy of science that seem to be prevalent in the quantitative-qualitative debate today (see Phillips, 1990, for a similar work covering largely different ground). I should begin, however, with a number disclaimers. Fit, I do not purport to present any original philosophical research in this article; my own quasi-philosophical views The author thanks Lee Sechrest, who served as action editor for this article, and Terry Hedrick and Ernie House for providing helpful comments on previous drafts. Both Steve Fuller and Chip Reichardt made a number of detailed comments that they generously allowed me to incorporate directly into this article. Of course, none of these readers agreed with all the material in this manuscript. Requests for reprints should be sent to William R. Shadish, Department of Psychology, The University of Memphis, Memphis, TN 38152.